Natural language understanding (NLU) refers to the technology that allows computers to understand, or derive meaning from, human languages. In general, NLU systems determine meaning from text. The text can be derived from any of a variety of sources, e.g., speech recognized text, keyboard entry, or the like. The meaning of the text can be determined and provided to other systems.
Some NLU systems determine meaning from text through a process that extracts information from the text. This process involves identifying features within the text as well as determining values for those features. Consider the text input “I want to order call forwarding for my home phone.” Determining the meaning of this text input can involve identifying three features including an intent feature, a service feature, and a context feature.
In this example, the value of the intent feature is “place order.” Other examples of potential values for the intent feature can include “cancelling a service,” “paying a bill,” or the like. The service feature is “call forwarding” and is the item that is being ordered. Other values for the service feature can include caller ID, voicemail, etc. The context feature is “home phone,” which identifies the particular account or context to which the service is being ordered. Determining values for each of these features can convey the meaning of the text input.
An NLU system that must extract information for even a small number of features still must be able to distinguish between a significant number of possible feature-value combinations. Consider an NLU system configured to identify and determine values for four features A, B, C, and D. Assume that feature A can take on any one of five values, feature B can take on any one of 10 values, feature C can take on any one of 20 values, and feature D can take on any one of 300 values. This leads to 5×10×20×300=300,000 possible combinations of values. Each combination of values can be referred to as a complex information target. Each complex information target indicates a particular meaning for text.
Often, many, if not all, of the features are independent of one another. This independence is demonstrated by the situation described above in which 300,000 possible complex information targets exist within the NLU system. Another attribute of independence is that not all features may be present in a given text input, which can complicate meaning determination.
One conventional approach to determining meaning from text within an NLU system is to build a statistical model that is trained on training data including text that is classified according to the total number of potential complex information targets. With respect to the example including features A, B, C, and D, each item of the training data would be classified into one of the 300,000 potential complex information targets. While the total number of complex information targets can, in some cases, be reduced, the number of complex information targets within a system even after target reduction still can number in the thousands.
The ability of a statistical model to accurately determine meaning, e.g., determine a correct complex information target for a given text input, decreases as the number of complex information targets within the NLU system increases. With larger numbers of complex information targets, overlap in meaning may also occur between different complex information targets, which further can reduce NLU system accuracy.